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Abstract 

Let K be a number field, let M. be the Hilbert modular orbifold of K, and let m q be the 
probability measure uniformly supported on the cusp cross sections of M. at height q. 
We generalize a method of Zagier and show that m q distributes uniformly with respect 
to the normalized Haar measure m on M as q tends to zero, and relate the rate by which 
m q approaches m to the Riemann hypothesis for the Dedekind zeta function of K. 
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1. Introduction 

Let J#2 = {% + iy € C : y > 0} be the Poincare upper half-plane with the hyperbolic 
metric ds 2 = (dx 2 + dy 2 )/2. The group PSL(2,R) acts on by fractional linear 
transformations which, as well, are hyperbolic isometries. The modular group T = 
PSL(2,Z) is a discrete subgroup of PSL(2,R) and the quotient space J^/T is the 
classical modular orbifold. From the classification of horocycles it follows that for each 
y > 0, the modular orbifold has a unique closed horocycle C y of length y . Let m y be 
the probability measure uniformly supported on C y (w.r.t. arc length). Let m be the 
normalized hyperbolic measure of We have the following well-known result due 

to D. Zagier (cf. (2^). 

Theorem 1. Let f be a smooth function on with compact support. Then, 

m y (f) = m(f) + oiy 1 ' 2 ^) (y -> 0) 

for all0<e< 1/2. 



In particular, the measures m y converge vaguely to m as y — > 0. Besides, in \22] Zagier 
establishes the following remarkable equivalence to the Riemann hypothesis. 
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Theorem 2. The Riemann hypothesis holds if and only if for every smooth function f 
with compact support on J^/F one has 

m y (f) = m{f) + o(y 3 ^) (y^O) 

for allO<e< 3/4. 

In the related work [l4[, P. Sarnak proved an analogue of these theorems for the unit 
tangent bundle of the modular orbifold, which can be identified with the quotient space 
PSL(2,M.)/T. Likewise, in [19( A. Verjovsky has shown that the analogue estimate of 
Theorem[T]in Sarnak's work is optimal for certain characteristic functions on PSL(2, K) /T. 

The purpose of this article is to apply Zagier's theory to the case of a number field. Let 
us briefly describe our results. Given a number filed K of degree n = r\ + 2r2 with ring of 
integers o, there exists a Riemannian manifold H — (J^Y 1 X (J&3Y 2 where the Hilbert 
modular group T = PSL(2, o) acts properly and discontinuously. If the field K has class 
number h, then the Hilbert modular orbifold ^# = H/T has h cusps and each cusp can 
be parametrized by the standard cusp at infinity. From the geometry of it follows 
that, for each cusp A and q > 0, there exist a generalized closed horosphere B(q, X)/T\ 
of dimension 2r\ + 3r2 — 1 and volume <z _1 c, where c is a certain constant depending on 
the field K. Let dv\> be the volume element of B(q, A) and m(q, A) be the probability 
measure in uniformly supported on B(q, X)/T\ with respect to dv' x . Let m be the 
normalized Haar measure of and m q — (m(q, Ai) + ... + m(q, Xh))/h. We have 

Theorem 3. Let f be a smooth function on ^ with compact support. Then, 
m q {f)=m{f)+o{q 1 ' 2 ^) (q 0) 

for all0<e< 1/2. 

We also give a generalization of Zagier's equivalence to the Riemann hypothesis and 
prove the following assertion. 

Theorem 4. The Riemann hypothesis for the Dedekind zeta function o/K holds if and 
only if for every smooth function f with compact support on ^# one has 

m g (/) = m(/) + o(g 3 / 4 - e ) (q 0) 

for all0<e< 1/4. 

We prove these theorems following the work of Zagier and Sarnak. We study the Mellin 
transform of q~ 1 m q and relate it to the Eisenstein series of the Hilbert modular orbifold 
by means of the Rankin-Selberg theorem. The Eisenstein series, being divisible by the 
Dedekind zeta function (k(s) of the field, enables us to translate properties of Ck(s) into 
properties of the Mellin transform of q~ 1 m q and, hence, properties of m q . 
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2. The Hilbert Modular Group 



In this section, we present the theory of the Hilbert modular group. We quote the 
classical books by Siegel [l6} , van der Geer Q , Freitag 0] , Elstrodt [B[ and the article by 
Weng [20] for a more comprehensive introduction to Hilbert modular orbifolds. 

Let J£§ ~ { z — (x,y) : x & C, y > } be the upper half-space in Euclidean three-space 
with the hyperbolic metric ds 2 (z) — (d\x\ 2 + dy 2 )/y 2 . As usual we identify a point 
z £ 3%% with an element of Hamilton's quaternions and write z = (x, y) = x + yj. The 
group PSL(2, C) acts by isometries on as follows: for a point z € J£§ we have 

z H> g ■ z = (az + b)(cz + d)^ 1 g = ± 

The hyperbolic volume element is equal to dxdy/y 3 where dx is the two dimensional 
Lebesgue measure on C. 



a b 
c d 



Let K be an algebraic number field with r\ real places and r 2 complex places and write 
n = [K : Q] = n + 2r 2 for the degree of K over Q. Consider the space H = J^ 1 x 
and denote a point in H by z = (zi, z r ), where r = r± + r%. The space T-L has the 
Riemannian metric 

r 

ds 2 (z) = Y / (d\^\ 2 + dy 2 )/yf 

i=l 

with corresponding volume element dv(z) = dxdy jy\ ■ ■ -yf, where dxdy is the Lebesgue 
measure on (W 1 xC 2 )xl r . The group G = PSL(2, R) ri x PSL(2, C) r2 acts transitively 
on % by isometries: 

9 ■ z := (ffi ' zi, -,9r ■ z r) 

where g = (gi, ■■■,g r ) G G. The isotropy group at the point (i, € H is the (maximal) 
compact subgroup PSO(2, M.) Tl x PSU(2) T2 and we have a natural identification between 
1-L and the quotient of G by this subgroup. 



To the real and complex places of K there correspond r embeddings of K into C. We 
arrange these embeddings as usual, and write them as a™, for a € K. Alike, we often 
write a = The group PSL(2,K) can be embedded in G as follows 



± 



a b 
c d 



^ ± 



(i) fed) 



,...,± 



Let be the ring of algebraic integers of K. The Hilbert modular group for the field 
K is the group F = PSL(2, 0) contained in PSL(2, K) C G. The action of F in U is 
irreducible and, since is a lattice, T is a discrete subgroup of G. Then a theorem of 
Thurston Ch. 13] implies that T acts properly and discontinuously on % and that 
the quotient space 

M = T\H 



is a difierentiable orbifold. It is called the Hilbert modular orbifold of the field K. 
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The action of G in % extents to the ideal boundary P = P(M) ri x P(C) r2 by fractional 
linear transformations. The cusps of T are the parabolic fixed points of T in P. They are 
the image of the application P(K) — > P given by ct/ (3 i— > (a^/[3^). The group T acts 
on the set of cusps, and a classical theorem of Maass, shows that non equivalent cusps 
are in one-to-one correspondence with the elements of the ideal class group of K. To the 
class of a fractional ideal a—(o-,p) C K there corresponds the class of the cusp X = a/p. 
In particular, to the identity in the ideal class group of K, that is to say, to the class of 
principal ideals, there corresponds the class of the cusp at infinity (oo, oo) G P. Then, 
if h is the class number of K, we know that T has h incquivalent cusps. 



Let A G P(K) be a cusp and write A = a/p, a = (cr,p). Since p and a generate a, there 
exist £, i] G a -1 such that pr\ — cr£ = 1, and we have a matrix 



p z 

g rj 



G SX(2,K) 



with the property A -1 (A) = oo. The matrix A is called a matrix associated to A. This 
matrix depends on the representation of A and the choice of £, i]. However, any other 
matrix associated to A as above is equal to 



a (3 
a" 1 



for some a, j3 G K . 



Proposition 5. Let A G P(K) be a cusp and T\ = {7 e T \ 7A — A} be the isotropy 
group ofT at A. Then, the group T\ is described as follows 



T\ = < A 



A" 



e e 0' 



where o x denotes the invertible elements of 0. 



Let us describe the action of T\ on H. First, the fractional ideal a -2 is a free Z-modulc 
of rank n and, therefore, has Z-bases. Let ct\, a n be such a base and let N(a) be the 
norm of the ideal a. Likewise, from the Dirichlet's units theorem, there exist fundamental 
units £i,...,£ r _i G X , such that 

o x = {e'l 1 •••£*r 1 1 I fci,...,fcr_i €Z } x W, 

where W is the group of roots of unity contained in K. Let z = (z\, z r ) be any point 
in H and define z* = A _1 z. We write 

z* = (zt, ...,**), x* = K,...,3?(x*),3(x*)), and y* = (yj, ...,1/?). 

Set N(y*) = yi 1 ■ ■ ■ y^ r , where Ni is the degree of the embedding i. The local coordinates 
of z at A are defined to be the 2r quantities 



q, Yi, Y r _i, Xi, X r 
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determined by the relationships (compare to [2C 



In 



In 



-(r-i) 



In 



In 



.,(!) 



a 



.(i) 



7 (r-l) 
'r— 1 

(i) ■ 



9f(a£ p >) 



3?(al r) ) 
9(al r) ). 



Yi ' 

Y r _! 

Xi 

SR(X r ) 
3(X r ) 



N(o)N(y*) 



5 ln ■« / — 

2 



1 In "s=i 

2 



Lemma 6. Let z eV. be a point, £ = m\ct\ + ■ ■ • + m n a n € a 2 and £ = wej 1 
o x , imi/i w £ W. Then, under the modular transformation 



M 



e- 1 



£/ie ZocaZ coordinates of z become 

g,Yi + fci,...,Y r _i +fc r _i,X* +mi,...,»(X*) + m n _i,3?(X*) + rn„ 

where the column vector X* = (X*, 5f(X*), 3?(X*)) is described as follows: if we write 
the definition of the local coordinates in matrix notation as O(X) = x* with the column 
vectors X = (Xi, 3?(X r ), 3(X r )) and x*, t/ierc X* = _1 E 2 OX w/iere E is the block 
matrix 

r« ... 



E = 



: '•• 

■■• -9f(eW) 
■■• ^(eW) 3i(eM) 



Proof. This follows from 



(Mz)* - A^MAz* = (eV + CHy*) - ((e w ) 2 x* + C W , 



'ft*)- 



Remark 1. If the group of roots of unity is different from ±1, then all Galois 
embeddings of K are complex. Let w £ W be different from ±1, then the action of 
~w 



A 







A in % = (C x R + ) r2 can be written in the coordinates (x*,y*) = A z 



as the transformations (x*,y*) H> {(w^) 2 x* , yf) , where each is a root of unity 
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Definition 1. A point z = (x, y) in T-L is called reduced with respect to A if its local 
coordinates at X satisfy 

0<q, -i<Y 1 ,...,Y f ._ 1 ,X 1 ,...,K(X I .),9f(X I .) < \ 
and his coordinate x* belongs to a fundamental domain of the action of W in C 1 * 2 by 



As a result of lemma |H] the set J^a of reduced points (w.r.t. A ) in H is a fundamental 
domain for T\. Moreover, define the generalized horosphere at height q (or distance to 
infinity 1/q) of the cusp A by B(q,X) = {z € % | /x(A,z) = q}. Then, the action of T\ 
on Jf? reduces to its action on B(q, A) = {z G H \ fi(z, A) = q}. 

Lemma 7. Lef dv' x be the measure induced on B(q,X) by the Riemannian metric on % 
and let dv be the Haar measure on %. Let R be the regulator and let D be the absolute 
value of the discriminant o/K. Then, in the local coordinates of X, we have 



t fo A (z) = (ri+4r 2 ) 1/2 2 
where dXdY is the Lebesgue measure in 

dv(z 



2 n-T5,-l -a 



DRdXrfY 

r ~ 1 . Likewise, we have 
DRdXdYdq 



Proof. Since A is an isometry, Riemannian metric objects in the coordinates z can be 
described in the coordinates z*. Now, since the change between the X's and the x*'s is 
linear, we have dXi • • • dX r = (2 r2 N(a) 2 /v / D) dx* (see [HI). Then, the volume induced 
by the Riemannian metric on the submanifold (x*,y*), with a fixed y*, is given by 



2~ r2 VD dXi ■ ■ ■ dX d 



N(y*) 

On the other hand, from the definition of the local coordinates, if q = N(y*), we have 



r-1 



y* = ^exp(^2Y fe ln ef ) 



k=l 



^l/n 



r-1 
k=l 



Si) 



2Y k 



foralH = l,...,r. The application (y*) h-> (log y*) is an isometry between (RIj_, n[=i dy*/y*) 
and (R r , Y^i=i dti). The local coordinates of y* factor through a commutative diagram 



(Y,g)^(Y,logg) 



+ 

log 



where (/(Yi, Y r _i, logg)) . = (J2k=\ ^Y/c log £^ +^ logq) is a linear transformation. 

Then, the measure induced on the submanifold N(y*) = q by the measure Yll=i dy* /y* 
is equal to ( cf. [l3j ) 

(ri + 4r 2 ) 1 / 2 2 ri ~ 1 ReYi • • ■ c?Y r _i, 
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where dYi • • • dY r -i is the Lebesgue measure on l r 1 . Moreover, 

r 

q- 1 T^ 1 RdYi ■ ■ • dY P _idg = Y[dy t /yi. 

»=i 

From these results the lemma follows. 



Remark 2. For each q > the action of T\ leaves dv' x invariant and the quotient 
T\\B{q, A) has volume 

q- 1 uj- 1 2 ri ~ r2 (n + 4r a ) 1/2 VBR, 

where uj is the number of roots of unity contained in K. Alike, we have a covering orbifold 
application i\ : T\\H — > . // . which gives the immersion i\(T\\B(q, A)) — » 



We now construct a fundamental domain for the action of T in % and describe the 
topology of the quotient. Let A = £ G P(K) be a cusp and write a = (p,cr). For z E W 
we have defined the height of z at A as 

N(a) 2 N(y) 

u(A,z) = * 

| N (_ CTZ + p )| 2 



This is independent of the choice of p and o~ € K and, for any 7 € T, we have the 
invariance property /x(7(A), 7(z)) = /u(A, z). The proofs of the following propositions can 
be found in 

Proposition 8. There exists a positive number li, depending only on K, such that for 
z e "H i/ie inequalities p(X, z) > and p(r, z) > Zi /or A, r £ P(K) imply A = r. 

Proposition 9. There exists a positive number I2, depending only on K, suc/i that for 
z e "H i/iere errisis a cusp A smc/i i/iai /«(A, z) > £2- 



Let A be a cusp. The sphere of influence of A is defined by 

S x = { z G % I //(A, z) > p(r, z) Vr e P(K) } 

The invariance condition /x(7Z, 7A) = /i(z, A) implies 5 7 (a) — Tf{S\)- The boundary of 
Sx consists of pieces defined by equalities /i(A,z) = p(r, z) with A ^ r. Moreover, the 
action of T in the interior F° of 5a reduces to that of the isotropy group Ta at A, i.e, if 
z and 7Z both belong to F°, then 7A = A. Let ix : Ta\5a — > T\H be the natural map. 
From propositions [5] and [51 we have 

T\H = U a za(Fa\5a) 



where the union is taken over a set of h non equivalent cusps. 
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Let G\ be the intersection of the sphere of influence at A and the reduced points with 
respect to A. Then G\ is a fundamental region for the action of T\ in S\. Since ^# is 
connected, there exist Ai, ...Ah non equivalent cusps such that U , l =1 G\ i is connected and 
a fundamental domain for T. For T sufficiently large we have a compact orbifold with 
boundary: 

J T =r\{ze?i: /Lt(z, A) < T for all cusps A} 

The boundary of consists of h compact orbifolds i\(T\\B(A, r)) of dimension 2r\ + 
3r 2 — 1 and, as a topological space, r\"H has h ends: 

T\H = IW T (a# T x [0, oo)). 



3. Non holomorphic Eisenstein series. 

We now define the Eisenstein series of the Hilbert modular orbifold and state their 
properties. Investigations on the Eisenstein series have been quite extensive. We quote 
the classical references 11 1, 0,0] and the exposition 0]. 



First, recall that W (as a Riemannian manifold) have associated the Laplace-Beltrami 
operator: 

h 9x2 + °y 2 + ,ir + i Vl dx * m + dy2 Vi 9yi 

It is invariant under the action of G on functions; that is, for all g E G, we have 
A(/ ° g) — o for any smooth function / on %. In particular, since 

A(N(y) s ) = (n+4r 2 ) S ( S -l)N(y) s 

we have 

A(m(A, z)) = (n + 4r 2 )s(s - l),u(A, z) 

for all points A € P(K). 

Let A be a cusp of the Hilbert modular group. The Hilbert modular Eisenstein series 
associated to A is defined by the series 

E A (z,,s)= J2 M A '7z) s 

7er A \r 

where the sum is taken over any complete set of representatives of the classes r^\r. 
This is independent of A in an equivalence class of cusps. The series defining Ea(z,s) 
converges absolutely and uniformly for s in bands {<r\ > 5R(s) > <jq > 1} and z in every 
compact subset of %. Therefore, it defines a continuous function which is holomorphic 
on the half-plane 3?(s) > 1. Moreover, it represents a T automorphic form, that is 

E A (z, s) = E A (7z, s) for all 7 e T 
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and satisfies the differential equation 



AE A (z, s) = (n + Ar 2 )s(s - l)E A (z, s) 

Recall that o x acts on the set of pairs (a,/3) G K 2 by e ■ (a, j3) = (ea,ef3) and that 
two pairs in the same orbit are called associated. Let {Mj}j £ N be a complete set of 
representatives for the quotient F A \T and, let A be a matrix associated to A. Write 



A _1 M 



7j 6 J 



Vj G N 



Then, the collection {(jj,Sj) : j G N} is a complete set of non associated pairs of 
generators for the ideal o and, from the definition of /z(A,z), we have 

E A (z,s) =N(a) s V M(y) ' „ . 

(7,(5)£o 2 /o x 1 w " 

In order to complete the Eisenstein series, we need the partial zeta function of an ideal 
class A. It is defined by the series 



integral 



The series is absolutely convergent for 3t(s) > 1, and uniformly in 3?(s) > 1 + e. We 
define the completed zeta function of an ideal class A by A(s)£k(s, -4) where, as in [l2j ]. 



A(s) = 2- r2S Dt7r-¥r (0 1 r( s ) r 



By a theorem of Hecke, Ck(s,A) has a meromorphic continuation to the whole complex 
plane and satisfies the functional equation 

&(s,A) = &(l-s,A'), 

where AA' = [0] is the ideal class of the different ideal of the field K. Recall that, by 
definition, is the dual module of o, w.r.t. the trace function of K. 

For a cusp r denote by A\ the ideal class associated with r. Let A be a cusp, the 
associated normalized Hilbert modular Eisenstein series is 

E A (z,s) = ^Ck(2s,^ a Uv)Ev(z, S ) (z G H, > 1) 

A' 

where the sum is taken over a set of non equivalent cuspsl. Writing the integral ideals 
of an ideal class C = [c] as the set of £c with £ G c _1 /o x , one can show that 

E A (z, s )=A(2 s )N(a) s ]T '"^ 



. ,, _ x |N( 7 z + ^' 

(7,5)^0 
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Let A' £ P(K) be another cusp and write A' = cr'/p', a' = (a 1 ' p 1 ). Let 



A' = 



(f e 

a' 77' 



be a matrix associated to A'. Since Eisenstein series are L-automorphic functions, in 
particular, we have that 



E£(A'z,a) = A(2s)N(a) s 



E 



N(y) s 



( 7 ,<5)eK 2 /° x 

0^(777' — So-' ,7^' — <5p')Ca 



|N( 7 z + ( 5)| / 



is invariant for the lattice W = a' C 



' — n'- 2 (- Ttt''l v (T r 2 



Hence, it has Fourier expansion 



e a (a' z ,,s)= a f' x (y> s 



where b'* = {£ € K | Tr(£o) c Z} is the dual module of W and 

Tr(x) = J2 Xi+ H 23?(Xl) ' 



i<ri i>r*i 



for x = (xi,...,x r ) £ M ri x C" 2 . In order to express the Fourier coefficients of the 
Eisenstein series we need the MacDonald Bcssel function: 

K s (y) = \J~ e-'t'+'-'J/V * for s G C, y > 0. 

The Bessel function has many properties (cf. [l[): 

i) it is invariant under the transformation s t— > —s, that is K s (y) = K- S (y). 

ii) all derivatives of K s (y) with respect to y are of rapid decay when y — > +00, i.e., 

K«Hy))\ <X§ s) (2)e-« > 2 1 = 0,1,2,... 

iii) it satisfies the differential equation 

y 2 ^ + y^--(y 2 + s*)\K s (y)=0. 



dy 2 dy 

iv) it is related to a Fourier transform, i.e., for I £ K/{0} and !R(s) > 1, we have 



y s 7r- s T(s) 



{t 2 +y 2 ) s 



dt = 2\ 



JyK s i{2-K\l\y) 



The following' theorem on the Fourier expansion of the Eisenstein series is proved by 
Sorensen in 01 (see also [13, S). 
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Proposition 10. Let b'* be the dual module of b 1 = a' 2 . Write z* = (x*,y*) = A' 1 z 
and qy = /z(A',z). Then, E^(z,s) has the Fourier expansion at the cusp A' 

E A (z,s) = CJ(2s,4Uv)^ + &(2«- IMa 1 ^ 1 )?^' 
+ 2"^ 2 J] rM s (0^-i/2(y*,0e 27riTr(xn) 



where 



T^ y (i)=N(b'di)- s/2 j2 



integral 
q:t>'0Z 



and 



tf,(yV) = tf.M Z« )...tf a (47rtf Z (r) ) 



Summing the partial Eisenstein series over a set of non equivalent cusps we have 
E(z, a) = ^ E A (z, s), E*(z, a) = 53 E$(z, s) 

A A 

The series E and E* are related by E*(z,s) = ££(2s)E(z, s). By exploring the Fourier 
expansion of E*(z, s) at the cusp A' = oo we can see that it represents a meromorphic 
function which is holomorphic in C except for two simple poles at s = 0, 1 and satisfies 
the functional equation 

E*(z,s)=E*(z,l-s) Vze-H VseC\{0,l}. 

From the class number formula, the residue of E*(z, s) at the simple pole s = 1 is equal 
to 2 Tl ~ 1 Rhuj^ 1 . Besides, counting the poles and zeros of E*(z, s), we can see that the 
poles of E(z,s) are in the zeros of the function ££(2s). Since, from the Euler product 
representation of the Dedekind zeta function and the Landau prime number theorem, 
Ck( s ) does not vanish in the region jft(s) > 1, we have that E(z, s) is holomorphic for 
3f (s) > 5 except for a simple pole at s = 1 with residue: 

Lemma 11. The Riemann hypothesis for the Dedekind zeta function Ck(s) holds if and 
only if f or all z & H the function E(z,s) is holomorphic in the half plane 3?(s) > 1/4, 
except for a simple pole at s = 1 . 

Proof. Suppose that E(z, s) is holomorphic in the half plane 5R(s) > \ for all z e W 
except for a simple pole at s = 1. Let s = a + it be such that 1/4 < a < 1/2 and 
Ck(2s) = 0, then Ck(^ s — 1) 7^ and the zero Fourier coefficient of E*(z, s) does not 
vanish. Therefore, there exists z e W such that E*(z, s) = (^(2s)E(z, s) ^ and so 
a = 1/4. The converse claim is clear. 
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Corollary 3.1. Let X be a cusp, let a be the ideal associated to X and b = a~ 2 . Let 
cj\ = A) and z* = A (z). Then, the function E(z, s) has the Fourier expansion at 
the cusp A 

or 1/2 

+ c*(2s) l^ T i- 2 *V) K *-VW ' l ' e 
^ ' leb* 



where 



t s (1) = N(f»W)- s/2 J! W 

q integral 
q-.bOl 



In addition, it satisfies the differential equation 

AE(z, s) = (n + 4r 2 )s(s - l)E(z, s) 

Proof. The Fourier expansion of E(z, s) at the cusp A' follows by dividing the Fourier 
expansion of E*(z, s) by C^(2s). Moreover, the differential equation 

A(N(y) s ) = (n+4r 2 ) S ( S -l)N(y) s 

and the differential equation of the Bessel function (together with its rapid decaying 
property) gives the differential equation for the analytic continuation of the Eisenstein 
series. 



Remark 3. The function 

&(2s~l) 

which appears in the zero coefficient in different cusps, is of greatest importance as it 
governs many properties of the Eisenstein series. 

4. Distribution of cusp cross sections 

In this section we prove our statements. We first show the Rankin-Selberg unfolding 
trick for the Hilbert modular orbifold and the Maass-Selberg relation for the Eisenstein 
series E(z, s). Distribution of the long closed horocycles and horospheres has been the 
subject of many works. For a dynamical point of view see (l9| and Q. 

Let a > be an integer or infinity. We denote by C"(./#) the set of complex valued 
functions defined on ^ of class C a with compact support. Also C{y^) denotes the set 
of continuous complex valued functions on For the rest of the section we fix a set of 
non equivalent cusps {A^} in order to simplify the arguments below. 
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Definition 2. Let X, Y, q be the local coordinates of z at the cusp \. For f G C(^#) 
let f be a function in the variables X, Y,q such that /(X, Y, q) = /(z). The measure 
uniformly supported at the cusp cross section at height q is defined by 

i i 

mi(f,q) = T ... T f(X,Y,q)dXdY, 
where dXdY is the Lebesgue measure on (W 1 x C 1 ' 2 ) x M r ~ 1 . 



For / G C°(^#) and each i G {1, h} consider the Mellin transform of mj(/, q)q 1 : 

Mi(f,s) := ( X m t (f,q)q s - 1 ^. 

Jo 1 

Proposition 12. For each f G C°(^#) and « G {1, /i}, i/ie integral defining A4i(f, s) 
converges absolutely in the half-plane Sft(s) > 1 and uniformly in strips of the form 
a i > 5i(s) > cr > 1- Therefore, it defines a holomorphic function in the half plane 
> 1. 

Proof. From the topology of the Hilbert modular orbifold it follows that a continuous 
function / belongs to C"(^#) if and only if the corresponding T-invariant function / 
in H is of class C a and there exists a constant T > such that if z G H satisfies 
M (z,A 4 ) > T, then /(z) = 0, for all i = {l,...,h}- Let H/IL = sup z£ ^{|/(z)|}. Then, 
since rrii(f,q) < H/H^, for 3?(s) > 1 we have: 

*)l < II/IL (^) . where a = 

Therefore, we have absolute convergence in Sft(s) > 1 and uniform convergence in strips 
of the form a\ > 5i(s) > ctq > 1- 



The following theorem is the Rankin- Selberg method for the Hilbert modular orbifold. 

Proposition 13. Let f G C°(^#) be a continuous function of compact support. Then, 
if^St(s) > 1, we have 

u- 1 2 r i- r *RVVM i (f,s)= f E Xi (z,s)f(z)dv(z) 

Proof. Let s be a point in the half plane 3ft(s) > 1 and let / be of compact support. 
Then 

f E Xi (z,s)f(z)dv(z)= I M(7z,A J ) s /(z)dt;(z) 

J m 7€ r x \r 

Since the measure dv is invariant, changing for a fundamental domain F, we can see 
that the last series is equal to 

V f M (z,A,) s /(z)rf W (z) = / M (z,A,) s /(z)rf W (z) 
76 r A .\r^( F ) Jr^XH 
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Then, if we use the local coordinates of z at the cusp Xi to evaluate the integral, we have 
/ M (z,A a ) s /(z)dt;(z) = V / / •••/ q s - 2 f(X,Y,q)dXdYdq 

= uj- 1 2 ri - r2 RVDM l (f,s) 

We shall need to estimate the Eisenstein series. Let h(K) be the number given by 
proposition [51 let T > h(K) be a fixed large constant. Define 



E T (z,s) 



E(z, s) if q = /i(A, z) < T for all cusps A 

E(z, s) — q s — 4>(s)q 1 ^ s if q = /i(A, z) > T for some cusp A 



The following theorem is the Maass-Selberg relation for the Eisenstein series of the Hilbert 
modular orbifold. We give a proof based on the unfolding trick (cf. 15, pp. 672]). 

Proposition 14. Let s, s' be two complex numbers such that s =/= s' s + s' 1. Then 

' T s+s'-i _ ^(^(s'jT 1 - 8 -*' \ 



/ E T (z,s)E T (z,s')dw(z) =C 



s + s' - 1 
T s ~ s '(/){s') -T s '- S <j)(s) 



+ C 

\ s — s 



where C = T^^Rhi 



Proof. First, from the Fourier expansion of E T (z, s) at the cusps we see that for each 
cusp A it decays quickly when q\ — ¥ oo and uniformly in s in vertical bands of finite 
width. Therefore the left hand side of the above equation is holomorphic in s and s' in 
the regular region of Eisenstein series. Now, for a real number T define 



S T (q) = 



1 if q > T 
if q < T 



Let F be a fundamental domain for the action of the modular group in %. For each 
i = 1, h and T large, we have an "end" of F: 

FMzef : %>T}, 
where qi = /i(z, \). With this notation, if 5R(s') > 3?(s) + 1 > 2, we have 

/ E T (z,s)E T (z,s')^(z) = / E(z,s)E T (z,s')du(z) 

r h 

= / E(z, a) (E(z, a') - V <J T (?i)9i ) dv(z) 

- r 

-^')E / E(z, S )g 4 1 " s dv(z), 



i=l " r T 
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These last terms can be calculated separately as follows. First, recall that for 5ft(s) > 1, 
we have the bound |E(z, s)\ < E(z, a). Then 

~ h 

/ E(z, a) (E(z, a') - V S T ( qi ) qf) dv(z) = 

P h hp 

/ E(z, a) (E(z, a') - ]T qf) dv(z) + ]T / E(z, s)qf dv(z) = 

JF i=l i=l 

hp hp 

£ / E(z, a) £ M (A« , 7 z) s ' <M») + J] / . E ( z , ' <M Z ) = 
i=i 7 er A .\(r-r x .) »=i jF ' F i 

h . h „ 

V/ E(z,a)q( dv(z)+J2 E(z,a)qf dv(z), 

, -, JFx.-F , , JF-Fl 



where is a fundamental domain for the action of T,^ in Since the disjoint union 
of — F and F — J 7 ^ is equal to 

{z : g 4 < T}HF Ai , 
the last equation is equal to 2 ri ~ r2 vTJRw -1 times 

h /.i /.i h /.T 

It 



• • ■ /" E ( X - Y » »)«'' dXidYidqi/q* = £ jTfo" + 0(a)gj-) gf' ^ 



a + s' — 1 s' — s 



On the other hand, the remaining term is equal to 

h „ h 



/ E(z,%^'^) = -£X>(s') I™ ^i+mq\~ s )q\- s '% 

i=i Jf t h i=i -/t 3j 



1 - S - S' 



where C = 2 ,1_r! \/DR/iw _1 . This is valid for 5J(s') > SR(s) + 1 > 2. Since both sides 
of the equation are meromorphic, the identity follows for s and a' in the regular region 
of both sides. 



Remark 4. If we replace E(z, s) with the constant function / = 1 in the procedure 
above, we can evaluate (see [l5[ and also 23| [20j |) 



E(z,s')dv(z) = C 



T 



s'-l 



(s')T- 
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for K(s') > 1, where C = 2 ri_r2 \/DR/iti) _1 . Since both sides are meromorphic functions, 
by analytic continuation, this identity holds for any s' in the regular region of both sides. 
Then, if we take the residue at s' = 1 and let T tend to oo, the volume of the Hilbert 
modular orbifold can be computed to be 

uoZOf) = 2- 3r2+1 7r- n D 3 / 2 C K (2). 
For each q > we have the measure 

h 

i=l 

and the Mellin transform of m q (f)q~ 1 : 

M(f, s) := r m q (f) <f 1 ^ = i]T s). 
Jo q i 

Then, from the Rankin-Selberg theorem, for / G C°(^#) and 5R(s) > 1, we have 

w- 1 /i2 ri - r2 R\/rj M(/,s) = f E(z, s) /(z) dv(z) 

Therefore, for a continuous function / with compact support, the Mellin transform 
M(f,s) has the same properties of E(z,s). That is to say, VW/(s) has a meromorphic 
continuation to the whole complex plane that is regular for 5R(s) > i except, possibly, 
for a simple pole at s = 1 with residue 

^ 1 (^ / ( S )) = ^/^/(z)d,(z). 

From Lemma QTJ it follows that the Riemann hypothesis for the field K holds if and only 
if for all / G C°(.-#) the function Mf(s) is regular for 5i(s) > 1/4 except, possibly, for 
a simple pole at s — 1 with residue given as above. 

Remark 5. The modified function: 

M* f (a) = &(28)Mf(a) 

has a holomorphic continuation to the whole complex plane except, possibly, for simple 
poles at s — 0, 1 and satisfies the functional equation 

M}{s) =M}(l-s). 

Now, let us recall the following facts about the order of growth of Ck(s) along vertical 
lines. For each a we define a number /J,(a) as the lower bound of the numbers I > such 
that 

Ck(o- + it) = 0(\t\ l ) as \t\ -> oo. 

Then fi has the following properties [H, P- 266]: 
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i) fi is continuous, non-increasing and never negative. 

ii) /i is convex downwards in the sense that the curve y = /x(cr) has no points above the 
chord joining any two of its points. 

hi) /Li(er) = if a > 1 and /i(cr) = n(h — a) if a < 0. 



Lemma 15. Let f be a differentiable function of compact support in ' . Then, for each 
1/2 < (To < I, there exists to such that 

(r 1+ 4r 2 )\M f (s)\<M^- fort>t a ,e>0 
\s(s- 1)| 

and (To < Sft(s) < 2, where /3/(<To) *s a constant depending on <7o fl^rf fie second derivatives 
off- 



Proof. From proposition ([13]) we have 

M(f, s)= J^E(z,s)f(z)dv(z). 

Since AE(z, s) = (r\ + 4r 2 )s(s — l)E(z, s) and A is a symmetric operator, then 

(n + 4r 2 )s(s - 1)M(/, s) = / AE(z, s) /(z) dv(z) 

E(z,s)Af(z)dw(z) 

Now, let T > be such that suppf C { qj < T }. Thus, 

(ri + 4r 2 )s(s - 1)X(/, s) = y E T (z, s) A/(z) dv(z) 

Therefore, the Cauchy-Schwarz inequality implies 

(r 1 + 4r 2 )| S (s-l)||^(/,s)|<||A/|| 2 ||E T (-, a )|| 2 . 
Now, from the Mass-Selberg relations, the L 2 norm of the Eisenstein series is given by 



/ \E T {z,s)\ 2 dv{z)=C 



c 



2(7- 1 

T 2it (j){s) - T~ 2it cj>(s) 
2it 



where s = a + it, t ^ 0, a =^ ^ and C = 2 ri ~ r ' 2 R\/D huj^ 1 . Therefore, we can bound the 
L 2 norm of the truncated Eisenstein series in vertical bands of finite width essentially by 
4>(s), except for a = 1/2 and t = 0. From Stirling's formula (cf. 21, Ch. 13]), it follows 
that 

T(a + it)~ V2^\tr 1/2 e-^/ 2 
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as t — > ±00 uniformly in vertical bands of finite width. Thus, 

7T*D~i \t 



2^7rf r(a - |) ri r(2s - 1)^ _„^_i u| _ n/2 



/ Dr(s) i 'ir(2s)'-2 

as < — > ±00 uniformly in 1/2 < 5R(s) < 2. On the other hand, if I > n/2 = /i(0), then 

( K (2s-l) = 0(t l ), 

uniformly in 1/2 < Jft(s) < 2. Furthermore, if <jq < 5i(s), thus 

1 = \ - M(g) 
Ck(2s) ^ N(ap 

where /i(a) is the Mobius function of the field K. Therefore, for all e > 0, we have 

\Hs)\ = o(\tn 

as \t\ — > 00 uniformly in co < 3?(s) < 2, which proves the claim. 

PROOF of theorem [31 Let / e Cf(^£). Then, from the Ranking-Selberg represen- 
tation, the Mellin transform A4(f,s) of m q (f)q~ 1 is holomorphic for 5R(s) > | except, 
possibly, for a simple pole at s = 1 with residue 

From proposition [T2] the Mellin inversion formula applies, and we have 

m q {f) = — / Mf^q^ds. (1) 

for any real number 6 > 1. Now, let < e < ~. Then, by the estimates of Lemma IT"5l 
we can shift the path of integration in equation (fTJ to the line a = \ + e to get 

m q (f) = m(f) + —J Mf(- + e + it)q^q~ u dt. 

Now, since M.f{\ + e + it) is integrable (w.r.t. dt), the Riemann-Lebesgue theorem 
implies 

M f {- +e + it)q lt dt 



lim 

g->0 



"2 







Therefore, 

m q (f) = m(f) + o(q 1 / 2 -*) (g->0). 

Now we show the relation with the Riemann hypothesis of the Dedekind zeta function. 
First, we state a classical consequence of the Riemann hypothesis (cf. [12, P- 267]). 



18 



Proposition 16. If the Riemann hypothesis for the Dedekind zeta function o/K is true, 
then we have the following estimates 

For e > and a > — : 
2 

-elogt < log|Cfr(s)| < elogt; s = a + it, t>t (e), 

that is to say,l 

'Ck(s)=<W 

< for every e > 0, s = a + it, a > \ as \t\ — > oo. 

Lemma 17. If the Riemann hypothesis for the Dedekind zeta function holds, then for 
each 1/4 < (To < 1/2, there exists to > such that 

/or all s with o$ < 5i(s) < 2 and 3?(s) ^ 1/2. .ffere /3/(o"q) is a constant depending 
essentially on ao and a finite number of derivatives of f. 

Proof. First, we estimate (f>(s) = Ck(2s — l)/(^(2s) in the region cr < $t(s) < 2, under 
the assumption of the Riemann hypothesis for the Dedekind zeta function Ck(s). From 
proposition [161 for every e > 0, £k(2s) _1 = 0(t e ), uniformly in (To < 5R(s) < 2. Alike, 
C(2s - 1) = O(t') for I > n = ^M/ 2 ), uniformly in a < Sft(s) < 2. Therefore, Stirling 
formula implies that, for every e > 0, (f>(s) = 0(t n / 2+e ), uniformly in ctq < 3ft(s) < 2. 
Now we can use the process of Lemma [T5l to see that a sufficient degree of derivatives of 
/ ensures that, 

\M,(a)\=0(\a(a-l)\- 1 ) (W ^ oo) 
for 5ft(s) 7^ 1/2 and uniformly in ctq < 3?(s) < 2. This proves the assertion. 



PROOF of theorem [H Suppose that for all / € C^{M), we have the following bound: 

m q (f)=m(f)+0(q 3 ^) (q 0) 

for all < e < 3/4 and write m q (f) = m(f) + k(q). Let T be sufficiently large and such 
that m q (f) = for q > T. Then, 

M/(s)= C m q {f)q s - 2 dq 
Jo 



'Mf) + k(q))q s - 2 dq 
m(/)T s - 1 



s- 1 
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+ / k(q)q s - 2 dq 



Since k(q) — 0(q^~ e ), the last integral converges absolutely and uniformly in the half- 
plane 5i(s) > 7 + e, so it defines a holomorphic function in that half-plane. Therefore, 
A4f(s) is a holomorphic function in the region 3t(s) > \ +e except, possibly, for a pole at 
s = 1 with residue m(f). Thus, the Riemann hypothesis for the Dedekind zeta function is 
true. On the other hand, suppose the Riemann hypothesis for the Dedekind zeta function 
of K holds. Then, Ck(2s) does dot vanish for 5R(s) > 1/4 and Aif(s) is holomorphic for 
3?(s) > 1/4 except, possibly, for a simple pole at s = 1 with residue m(f). From Lemma 
IT5l the integral of M. f(s)q 1 ~ s exists over the boundary of the band | + e < a < 2, for 
all < e < 1/4. Therefore, the Mellin inversion formula implies 

1 f°° 1 

m q (f) = Kes a=1 (M f (s)) + — / Mj{- + e + it)g-**g* - e dt. 

27r J -co 4 

Again, by the Riemann-Lebesgue Theorem: 

m q (f)=m(f)+o(qi- e ) (q -> 0). 
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